Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial bpfman-operator support for Load/Attach Split #347

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Initial bpfman-operator support for Load/Attach Split #347

wants to merge 3 commits into from
+15,042 −5,110

Conversation

anfredette
Copy link
Contributor

@anfredette anfredette commented Dec 9, 2024 •
edited
Loading

Initial bpfman-operator support for Load/Attach Split with Initial Cluster-Scoped XDP Programs

NOTE: This code doesn't actually do the load/attach split, but simulates it be re-loading the
program for each attachment.

This commit introduces the foundation for the load/attach split, including:

  • Updates to the BpfApplication CRD to support a separate list of optional
    attach points for programs. This allows programs to be loaded before
    attachments are made and enables dynamic attachment updates.
  • An initial version of the BpfApplicationNode CRD to manage per-node
    information for a single BpfApplication.
  • Proof of concept and initial implementation for cluster-scoped XDP
    programs, with working unit tests in the app-agent.

Additionally:

  • Updated existing controllers to work with the new CRD format, though
    currently limited to XDP programs.
  • Initial changes in the operator to support the load/attach split, with
    further cleanup and support for additional program types planned.

TODO:

  • Generalize the agent code to support multiple program types.
  • Support for namespace-scoped CRDs and other program types beyond XDP.
  • More clean-up, complete testing, and address remaining edge cases, ...

@anfredette anfredette force-pushed the load-attach-split branch 2 times, most recently from e352df4 to 0531a5f Compare December 10, 2024 22:15
@codecov Codecov
Copy link

codecov bot commented Dec 10, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 43.40152% with 1411 lines in your changes missing coverage. Please review.

Project coverage is 31.00%. Comparing base (dd66d41) to head (69a1b95).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
apis/v1alpha1/zz_generated.deepcopy.go 50.33% 213 Missing and 9 partials ⚠️
controllers/app-agent/common.go 47.01% 144 Missing and 16 partials ⚠️
controllers/app-agent/application-program.go 56.62% 124 Missing and 20 partials ⚠️
controllers/app-agent/containers.go 0.00% 124 Missing ⚠️
controllers/app-agent/internal/bpfman-core.go 22.30% 103 Missing and 5 partials ⚠️
controllers/app-agent/xdp-program.go 53.55% 88 Missing and 10 partials ⚠️
controllers/app-agent/tcx-program.go 54.08% 80 Missing and 10 partials ⚠️
controllers/app-agent/internal/cmp.go 34.12% 67 Missing and 16 partials ⚠️
cmd/bpfman-agent/main.go 0.00% 63 Missing ⚠️
controllers/app-agent/internal/auth.go 0.00% 60 Missing ⚠️
... and 9 more
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #347      +/-   ##
==========================================
+ Coverage   28.12%   31.00%   +2.88%     
==========================================
  Files         128      148      +20     
  Lines       11207    13848    +2641     
==========================================
+ Hits         3152     4294    +1142     
- Misses       7773     9149    +1376     
- Partials      282      405     +123
Flag Coverage Δ
unittests 31.00% <43.40%> (+2.88%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anfredette anfredette force-pushed the load-attach-split branch 2 times, most recently from 113089c to 60d0115 Compare December 10, 2024 22:31
@anfredette
Copy link
Contributor Author

anfredette commented Dec 10, 2024 •
edited
Loading

All of the examples will need to be updated with the new CRD format before the Kubernetes integration tests will pass.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 18, 2024

@anfredette, this pull request is now in conflict and requires a rebase.

@mergify mergify bot added the needs-rebase label Dec 18, 2024
@mergify mergify bot removed the needs-rebase label Dec 18, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 9, 2025

@anfredette, this pull request is now in conflict and requires a rebase.

@mergify mergify bot added the needs-rebase label Jan 9, 2025
@mergify mergify bot removed the needs-rebase label Jan 9, 2025
@anfredette anfredette force-pushed the load-attach-split branch 2 times, most recently from 10366ab to 0c7f942 Compare January 9, 2025 14:57
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 22, 2025

@anfredette, this pull request is now in conflict and requires a rebase.

@mergify mergify bot added the needs-rebase label Jan 22, 2025
@anfredette anfredette changed the title WIP: Initial pass at BpfApplication CRD update for load/attach split WIP: New bpfman-operator design and CRDs to support the load/attach split Jan 22, 2025
@anfredette
Copy link
Contributor Author

This is still a WIP, but it's working for cluster-scoped XDP programs.

Here's a sample of the kubectl output from installing and deleting an XDP program:
https://gist.github.com/anfredette/4433aaa58518db5d4d14ed2bf4218f55

If you're looking at the code, the new operator and agent code is in the app-operator and app-agent directories, while the old code is still in bpfman-operator and bpfman-agent. I plan to eventually delete the old directories and rename the new ones, but I'm keeping the old code around for comparison and to pull from as needed.

@anfredette anfredette force-pushed the load-attach-split branch 2 times, most recently from 2799666 to d68ab2a Compare January 22, 2025 17:19
@mergify mergify bot removed the needs-rebase label Jan 22, 2025
@anfredette anfredette changed the title WIP: New bpfman-operator design and CRDs to support the load/attach split Initial bpfman-operator support for Load/Attach Split Jan 23, 2025
@anfredette anfredette force-pushed the load-attach-split branch 2 times, most recently from 203e4ff to 69a1b95 Compare January 24, 2025 22:28
@anfredette
Initial support for Load/Attach Split
9e5cb54 
Includes support for Cluster-Scoped XDP, TXC, and Fentry Programs

This commit introduces the foundation for the load/attach split, including:
- Updates to the `BpfApplication` CRD to support a separate list of optional
  attach points for programs. This allows programs to be loaded before
  attachments are made and enables dynamic attachment updates.
- An initial version of the `BpfApplicationState` CRD to manage per-node
  information for a single `BpfApplication`.
- Proof of concept and initial implementation for cluster-scoped XDP,
  TCX, and Fentry programs, with working unit tests and samples.

See TODO.md for more info.

Signed-off-by: Andre Fredette <[email protected]>
@anfredette
Copy link
Contributor Author

I've cleaned up the code a bit and added support for TCX and Fentry. See the TODO.md file for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@anfredette